Denoising Diffusion Autoencoders are Unified Self-supervised Learners

Inspired by recent advances in diffusion models, which are reminiscent of denoising autoencoders, we investigate whether they can acquire discriminative representations for classification via generative pre-training. This paper shows that the networks in diffusion models, namely denoising diffusion autoencoders (DDAE), are unified self-supervised learners: by pre-training on unconditional image generation, DDAE has already learned strongly linear-separable representations within its intermediate layers without auxiliary encoders, thus making diffusion pre-training emerge as a general approach for generative-and-discriminative dual learning. To validate this, we conduct linear probe and fine-tuning evaluations. Our diffusion-based approach achieves 95.9% and 50.0% linear evaluation accuracies on CIFAR-10 and Tiny-ImageNet, respectively, and is comparable to contrastive learning and masked autoencoders for the first time. Transfer learning from ImageNet also confirms the suitability of DDAE for Vision Transformers, suggesting the potential to scale DDAEs as unified foundation models. Code is available at github.com/FutureXiang/ddae.Comment: ICCV 2023 Ora

Faster Ray Tracing through Hierarchy Cut Code

We propose a novel ray reordering technique to accelerate the ray tracing process by encoding and sorting rays prior to traversal. Instead of spatial coordinates, our method encodes rays according to the cuts of the hierarchical acceleration structure, which is called the hierarchy cut code. This approach can better adapt to the acceleration structure and obtain a more reliable encoding result. We also propose a compression scheme to decrease the sorting overhead by a shorter sorting key. In addition, based on the phenomenon of boundary drift, we theoretically explain the reason why existing reordering methods cannot achieve better performance by using longer sorting keys. The experiment demonstrates that our method can accelerate secondary ray tracing by up to 1.81 times, outperforming the existing methods. Such result proves the effectiveness of hierarchy cut code, and indicate that the reordering technique can achieve greater performance improvement, which worth further research

Association between 25(OH)D Level, Ultraviolet Exposure, Geographical Location, and Inflammatory Bowel Disease Activity: A Systematic Review and Meta-Analysis.

There is no consensus on the vitamin D levels and inflammatory bowel disease (IBD).To conduct a systematic review and meta-analysis to analyze the relationship between IBD and 25(OH)D, sun exposure, and latitude, and to determine whether vitamin D deficiency affects the severity of IBD.We searched the PubMed, EBSCO, and ClinicalTrials.gov databases to identify all studies that assessed the association between 25(OH)D, sun exposure, latitude, and IBD through November 1, 2014, without language restrictions. Studies that compared 25(OH)D levels between IBD patients and controls were selected for inclusion in the meta-analysis. We calculated pooled standardized mean differences (SMDs) and odds ratios (ORs).Thirteen case-control studies investigating CD and 25(OH)D levels were included, and eight studies part of above studies also investigated the relationship between UC and 25(OH)D. Both CD patients (SMD: 0.26 nmol/L, 95% confidence interval [CI]: 0.09-0.42 nmol/L) and UC patients (SMD: 0.5 nmol/L, 95% CI: 0.15-0.85 nmol/L) had lower levels of 25(OH)D than controls. In addition, CD patients and UC patients were 1.95 times (OR, 1.95; 95% CI, 1.48-2.57) and 2.02 times (OR, 2.02; 95% CI, 1.13-3.60) more likely to be 25(OH)D deficient than controls. We also included 10 studies investigating the relationship between CD activity and vitamin D. Results showed that patients with active CD (CD Activity Index ≥ 150) were more likely to have low vitamin D levels. In addition, whether low sun exposure and high latitude were related to a high morbidity of CD need to be provided more evidence.Our study shows that IBD patients have lower vitamin D levels. For active CD patients, vitamin D levels were low. These findings suggest that vitamin D may play an important role in the development of IBD, although a direct association could not be determined in our study

Studies on vitamin D levels in CD patients, UC patients, and controls.

<p>*Vitamin D levels reported as nmol/L. Vitamin D levels measured in ng/mL were converted to nmol/L by multiplying by 2.496.</p><p>** Median and range</p><p>***NS, not significant</p><p>^#<i>P</i> compares CD patients or UC patients to controls.</p><p>Studies on vitamin D levels in CD patients, UC patients, and controls.</p

Meta-analysis of studies reporting 25(OH)D levels in CD patients vs. controls, a standardized mean difference with a 95% confidence interval and weight percentage.

<p>Subtotals of adults and children, and overall population.</p

Meta-analysis of studies reporting dichotomous outcomes of 25(OH)D levels in CD patients vs. controls and estimated OR with a 95% confidence interval and weight percentage.

<p>Meta-analysis of studies reporting dichotomous outcomes of 25(OH)D levels in CD patients vs. controls and estimated OR with a 95% confidence interval and weight percentage.</p

Data for vitamin D deficiency among CD patients, UC patients, and controls.

<p>* <i>P</i> compares CD or UC patients to controls.</p><p>** Median and range</p><p>***Because of some missing data on serum 25[OH] vitamin D levels, the number of deficiency and sufficiency together didn't match the total number.</p><p>^#NS, not significant</p><p>Data for vitamin D deficiency among CD patients, UC patients, and controls.</p

Meta-analysis of studies reporting dichotomous outcomes of 25(OH)D levels in UC patients vs. controls and estimated OR with a 95% confidence interval and weight percentage.

<p>Meta-analysis of studies reporting dichotomous outcomes of 25(OH)D levels in UC patients vs. controls and estimated OR with a 95% confidence interval and weight percentage.</p

Meta-analysis of studies reporting 25(OH)D levels in UC patients vs. controls, a standardized mean difference with a 95% confidence interval and weight percentage.

<p>Subtotals of adults and children, and overall population.</p